Merged
Conversation
- Rename/delete commands have 'when: false' in command palette, so executeCommand cannot find them. Use right-click context menu instead, matching the approach used in the old UI test suite. - Skip 'create with no build tools' test because scaffoldSimpleProject() opens a native OS file dialog that Playwright cannot automate. - Add selectContextMenuItem() helper to VscodeOperator. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The .action-item div is a presentation container; VS Code only handles click events on the inner <a role='menuitem'> anchor. Use getByRole to find and click the correct element, and wait for menu dismissal to confirm the click registered. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Playwright's click() does not reliably trigger actions on VS Code Electron context menu items (the menu stays open). Switch to: 1. Right-click to select tree item and set focusedView 2. Escape to close context menu (returns focus to tree) 3. F2/Delete keyboard shortcut to trigger rename/delete commands Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
VS Code menus internally handle mouseup events, not click events.
Playwright's simulated .click() in Electron does not reliably trigger
the mouseup handler. Use dispatchEvent('mouseup') to directly dispatch
the event that VS Code's menu system listens for.
Previous approaches that failed:
- .click() on .action-item container: menu stays open
- getByRole('menuitem').click(): menu stays open
- keyboard shortcut (F2/Delete): focusedView not set after Escape
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Neither .click(), dispatchEvent('mouseup'), nor keyboard shortcuts
reliably trigger VS Code Electron context menu items via Playwright.
Switch to raw page.mouse.click() at the element's bounding box center
coordinates, which sends CDP-level InputDispatchMouseEvent that
Electron's native event handling processes correctly.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…dialogs Root causes found and fixed: 1. Context menu click requires hover first to trigger VS Code's menu focus state (.action-item.focused), then click works reliably. 2. redhat.java shows an Electron native dialog (dialog.showMessageBox) for refactoring confirmation on file rename. Playwright Page API cannot interact with native dialogs. Monkey-patch showMessageBox in the Electron main process to auto-return OK. Verified locally: both rename and delete tests pass. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
On Linux the button order in dialog.showMessageBox differs from Windows, so response:0 selects 'Show Preview' instead of 'OK'. Fix by scanning the buttons array for the 'OK' label. Also handle the Refactor Preview panel (Apply/Discard) as a fallback in case the dialog still enters preview mode. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
On Linux CI (headless, no recycle bin), the delete confirmation uses 'Delete' or 'Move to Trash' as button labels, not 'OK'. Expand the regex to match all known confirm labels. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR migrates the extension’s UI/E2E testing from vscode-extension-tester to a Playwright-based Electron harness, adding new E2E operators and test suites and updating CI workflows to run them.
Changes:
- Add a Playwright Electron fixture + operators (generic VS Code + Java-specific) and initial E2E test suites.
- Introduce a Playwright config + global setup to provision VS Code and install required extensions/VSIX.
- Wire E2E execution into
package.jsonand switch Linux/Windows CI workflows to run the new Playwright E2E job and upload test artifacts.
Reviewed changes
Copilot reviewed 13 out of 14 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
tsconfig.json |
Excludes the new test/e2e tree from the root TypeScript build. |
test/simple/.project |
Updates Eclipse project metadata (formatting + filtered resources). |
test/e2e/utils/vscodeOperator.ts |
Adds generic VS Code UI helper operations for Playwright. |
test/e2e/utils/javaOperator.ts |
Adds Java-specific UI helpers (LS readiness, tree expansion, etc.). |
test/e2e/utils/constants.ts |
Centralizes E2E timeouts/roles/constants. |
test/e2e/tests/projectExplorer.test.ts |
Adds E2E coverage for Java Projects explorer behaviors. |
test/e2e/tests/libraries.test.ts |
Adds (currently skipped) E2E scaffolding for library/project creation flows. |
test/e2e/tests/fileOperations.test.ts |
Adds E2E tests for create/rename/delete operations in Java Projects view. |
test/e2e/playwright.config.ts |
Defines Playwright runner settings, reporting, artifacts, and global setup. |
test/e2e/globalSetup.ts |
Downloads VS Code and installs redhat.java (+ optional VSIX) pre-test. |
test/e2e/fixtures/baseTest.ts |
Implements the Playwright fixture that launches VS Code via Electron and manages per-test workspace setup/teardown. |
package.json |
Adds test-e2e script and introduces @playwright/test dev dependency. |
.github/workflows/windowsUI.yml |
Switches Windows CI job from old UI tests to Playwright E2E + artifact upload. |
.github/workflows/linuxUI.yml |
Switches Linux CI job from old UI tests to Playwright E2E + artifact upload. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Remove unused imports in fileOperations.test.ts and libraries.test.ts - Clean up skipped test bodies and dead beforeEach in libraries.test.ts - Fix JSDoc comment in vscodeOperator.ts to reflect actual CSS fallbacks - Increase Playwright timeout from 180s to 240s for LS readiness margin - Remove cliArgs spread from Electron launch (avoid CLI-only flags) - Remove VSIX install from globalSetup (extensionDevelopmentPath suffices) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
chagong
approved these changes
Apr 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.